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(57) Abstract 



. A digital video piesentation system (30) is provided with a decoder (56) to decode full frame MPEO-2 video by a single method 
mat applies regardless of buffer memory (42) and frame rate conversion considerations. A display control module (82) handles frame rate 
and field sequence in response to host configuration, particularly buffer memory (42) size and display type (NTSC or PAL), to host trick 
play command signals, and to information in the received bitstream. Pictures are decoded as buffer memory (42) for the decoded pictures 
becomes available, and picture display attributes are assigned and stored in a table (83), one string for each decoded picture. 



FOR THE PURPOSES OF INFORMATION ONLY 



Ah 
AM 
AT 
AU 
AZ 
BA 
BB 
BE 
BP 
BO 
BJ 



Codes used to identify 

Albania 



party to the PCT on the front pages of pamphlets publishing international applications under the PCT. 



Austria 

Australia 

Azerbaijan 

Bosnia and Herzegovina 



Burkina Paso 
Bulgaria 



CA 
CP 
CG 
CH 

a 

CM 
CN 

cu 
cz 

DB 



Brazil 
Belarus 



BS 

FI 

FR 

OA 

GB 

GE 

GH 

GN 

GR 

HU 

IB 

IL 



Finland 



Gabon 

United Kingdom 

Georgia 

Ghana 



Greece 

Hungary 

Ireland 



IS 
LT 
LU 
LV 
MC 
MD 
MG 
MK 

ML 
MN 
MR 



Lesotho 



L^embourg 



Republic of Moldova 

Madagascar 

The former Yugoslav 

Republic of Macedonia 

Mali 

Mongolia 



SI 


Slovenia 


SK 


Slovakia 


SN 


Senegal 


sz 


Swaziland 


TD 


Chad 


TO 


Togo 


TJ 


Tajikistan 


TM 


Turkmenistan 


TR 


Turkey 


XT 


Trinidad and Tobago 


UA 

Ufl 


Ukraine 


us 


Uganda 

Uohcd Stales of America 


uz 


Uzbekistan 


VN 


Viet Nam 


YU 


Yugoslavia 


zw 


Zimbabwe 



Canada 

Central African Republic 



Cdtedlvoire 
Cameroon 
China 
Cuba 

Czech Republic 



Denmark 



IT 
JP 

KE 
KG 



KZ 
LC 
LI 
LK 
LR 



Italy 

Japan 

Kenya 

Kyrgyzstan 

Democratic People's 

Republic of Korea 

Republic of Korea 

Kazakstan 

Sunt Lucia 

Liechtenstein 

Sri Lanka 

Liberia 



MW 

MX 

NE 

NL 

NO 

NZ 

PL 

PT 

RO 

RU 

SD 

SB 

SG 



Malawi 



Netherlands 
Norway 
New Zealand 
Poland 



Russian Federation 

Sudan 

Sweden 



WO 00/59219 



PCT/USOO/01023 



10 



- 1 - 

DIGITAL VIDEO DECODING BUFFFriwi- «un 
FRAME-RATE CONVERTING Smo ANdTpp^TUS 

This applied is reiated to the common* assigned and copending 

U.8. paten, appiicattons. each ofwhich ,s hereby expressly Incorporated by reference 
herein: 

Sena, No. 08*46,590, fifed Apr, 30, 1097, by Edward J. Paiuch. enfified Memory 
Address Generation For Digital Video; 

Sena, No. 08/865.749, Ned May 30, 1997, by Mosb. Subfile, a!., emitted Special 
Puipose Processor For Digital AudtoMdeo Decoding; 

Serial No. 08*66.419. Bed May 30. 1997. by Taner OzcefiK e« a,.. emfitedTaek 
And Stack Manager For Digital Video Decoding; 

Serial No. 09/001,122, Sled December, ,997. by Subrolo Bose e. al. emfited 
Mofion Compensated Digitel video Decoding w«h Buffered Picture Sterag. Memory 
Map; 

Serial No. 09/001,129, filed December ^0 1007. by S .,h^ „ , ^ , 



15 



Mofion Compensated Digitel Video Decoding and Buffer Memory Addressing 
Therefore; 



WO 00/59219 

PCT/USOO/01023 

-2- 

Seria. No. 09/177,261, filed October 22, 1998, by Cem Duruoz et a.., entitled 
Method And Apparatus For a Virtual System Time Clock For Digital/audio/video 
Processor; 

Serial No. 09/178,803, filed October 26, 1998 by Gem Duruoz et al„ entitled 
Management of Trick Playback of Digital Video Data; 

Serial No. 09/__ fi , e d concurrently herewith, by Cem Duruoz et al., entitled 
Trick Playback of Digital Video Data; 

Serial No. 09/__,_, file d concurrently herewith, by Fang-Chuan Wu, entitled 
On Screen Display; and 

Serial No. 09/____ filed concurrently herewith, by Fang-Chuan Wu; entitled 
Display Master Control. 
Field aOhs frr\"»nff~n 

The present invention relates to digital video presentation, and particularly, to 
systems and methods for converting frame rates of decoded MPEG video streams for 
display. More particularly, the present invention relates to the determination and control 
of frame conversion and field display sequencing in MPEG video receiving and 
presentation systems, including those of differing configurations. 
Background sfcffls Iraflnflap 

A standard for digital video and audio programs for broadcast and for recordings 
such as video compac t disks (VCD) has been established by the Motion Pictures Expert 
-coup uuiHt*,) ciiuft mu lv mu umnmum uiycii^iio.. ioi stanuaidLcUiuh U buj. 
Such standards for digital video and two channel stereo audio were established and 
known as MPEG-1, more formally, as ISO-11172. An enhanced standard, known 
colloquially as MPEG-2 and more forma.ly as ISO-13818. has been established to 
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provide for enhanced quality and for spacing date fcrmata for broadoaat and other 
higher noiae epplicationa es we» aa digital video diska (DVD) and other higher 
resolution recorded media. 

The MPEG video standard specifies a bitstream syntax that typically provides 
transformation blocks of 8 x 8 iuminance pels (pixeis) and corresponding chrominance 
data using Discrete Cosine Transform (OCT) coding. The OCT coding is performed on 
the 8x8 pel blocks followed by quantization, zigzag scan, and variable length coding of 
runs of zero quantized indices and amplitudes of the indices. Motion compensated 
prediction is employed. For video, MPEG contemplates Intra (I) frames, Predictive (P) 
frames and Bidirectional^ Predictive (B) frames. The l-frames are independently coded 
and are the least efficiently coded of the three frame types. P-frames are coded more 
efficiently than are .-frames and are coded relative tothe previously coded I- or Pfiame. 
B-ftames are coded the most efficiently of the three frame types and are coded relative 
to both the previous and the next I- or P-frames. The coding order of the frames in an 
MPEG program is not necessarily the same as the presentation order of the frames. 
Headers in the bitstream provide Information to be used by decoders to properly decode 
the time and sequence of the frames for the presentation of a moving picture. 

The video bitstreams in MPEG systems include a Video Sequence Header, 
which is the primary definition of the entire video sequence. The Video Sequence 
Header contains picture size and asp ect ratio data, bit rate limits and other global 
HA-cu^rs. , n dHLv^, vanous sequence bdensions may also be included that 
contain other information applicable to all pictures of the sequence, including a 
Progressive Sequence bit which indicates that the sequence contains only Progressive 
Frame pictures, a Chrominance Format code, information indicating the frame rate of 
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at which the original picture was encoded including original video format (e.g., NTSC, 
PAL, other) and other variables. Following the Video Sequence Header and Sequence 
Extension are coded Groups-Of-Pictures (GOPs), which are the components of the 
sequence that enable random access of the video stream. Each GOP usually includes 
5 only one l-picture and a variable number of P- and B-pictures. Each GOP also includes 
a GOP header that contains presentation delay requirements and other data relevant 
to the entire GOP. Each picture in the GOP includes a Picture Header, which is the 
primary coding unit that contains picture type and display order and delay data and 
other information relevant to the picture, including whether the picture is an /-. P- or 

1 0 B-p/cfty/e, whether the picture is a frame or a field picture, whether a frame picture is a 
progressive frame or interlaced video, whether the field is to be repeated (3:2 pull-down 
as described below), field display order and other parameters. 

Each MPEG picture is divided into a plurality of Macroblocks (MBs), not all of 
which need be transmitted. Each MB is made up of 16 x 16 luminance pels, or a 2 x 2 

1 5 array of four 8x8 transformed blocks of pels. MBs are coded in Slices of consecutive 
variable length strings of MBs, running left to right across a picture. In MPEG-2, slices 
may begin and end at any intermediate MB position of the picture but must respectively 
begin or end whenever a left or right margin of the picture is encountered. Each Slice 
begins with a Slice Header that contains information of the vertical position of the Slice 

20 within the picture, information of the quantization scale of the Slice and other 
fnft,mrat,on SOen BS mat whluh deih be used tor fast-forward, fast reverse, 
resynchronization in the event of transmission error, or other picture presentation 
purposes. The Slice Header primarily facilitates resynchronization. refresh and error 
recovery. 
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The Macroblock Is the basic Ml used to, MPEG motion compensation Each 
MB contains an MB Haader. which, for the first MB of a SB ce. contafos mfonnabon of 
the MB* horfconta, posaon re,a«,e to «he left ed 9 e of the picture, ana which, for 
subsequent^nsn,^^^,^^^^^^^^^^ 

co„sec U »»MBsofaS,caa ra be„sn,«edw B h»,eS« te .TheM B Header W . nM asa,a 
macroblock type, such as Inbafietd ptedicUve which Is restocted ,o only pels from the 

cunen. fmme. or Interfield ptedlafce which allows copyfog of pels .torn a pre,,ous 
*ame. Tbe MB header also defines Moflon Vector Type. DCTJype ffmme or «e,d 
OCT,, me motion vecto*. the blacks that are encoded end macmbtock parameters 
The individual 8X8 pa. blocks, fourofwhfch make up me macmbtook. have no headers 
and are the basic transform and compression unit 

The presentation of MPEG video involves the display of vtdeo frames ata rafo 
ef. for example, t^nty.^ or Wrly fames per ^ ^ ^ ^ 

afondard used. PAtor NTSC, forexempfo, Thirty foames per second corresponds to 
presemedon time Intervals of approximately 32 milliseconds. The cepeclty o, MPEG 
signals ,o carry Infom^don ,a achieved ,„ part by exploKing foe concept ma, mere is 
WcaHy a high degree of conelabon baween adjacent pictores and by expfoi.no 
temporal redundandas in the coding of the signals. Whene two conseeuflve video 
frames of a program ana nearly identical, for example, the communication of the 
■tonsecudveftames re quires, forexample, only the transmission of one before afong 

■■ m i m OBnsmsstBmsn i pmnmmm bhiy w ^.uMum^^^ 

l-P«ture. or Reference Pictore. along wim InfomWfon needed by me decoder a, me 
receiver to reconstruct the P-plcture from the previous l-plcture. This means that foe 
decoder must have provision for storage of the Reference Picture data. 
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Information contained In a FfUum transmission includes blooks otvldeo data 
no.con b ,nedi„ a Re fere nca l .orP^ a5TO , a8 ^ |n ^ 6nneededtocoi)y 

used in MPEG systems ,o accomplish P^ ^ , ^ 

■sbretechnloueo, Forwam Predicdon m which a Predledon in meform of , Predion 
Motion Vector Wtetransmilte<lln me video daBo, a given or Targe, MB .The 
MV .elis .he deooder where and how to exbac, a ,6x16 block o, pixe, data from me 
or P- Referee Ptatore to be reproduced as me Targe, MB. if needed, a Predion 
Enor is banamttted in the torn, of en enor block tha. contoins pixel data needed to 
supplement the copied modon compensated date in order to oomptete the ounent 
picture. 

With B-pictures. ma Bidirecdona, Temporal Predion technique called Modon 
Compensated Interpoiedon is used. Modon Compensated interpotadon Is 
accomplished by .ransmrrHng. in lieu of all of the video date tor a Targe, MB. an MV that 

sp^Bes^^xISb^ofplxebtocopyeimerfromd^p^ousRete^Pioto^ 
or from me next future Referee Plctore, or from the average of one ,6x16 block o, 
prxeis bom each of the previous and next Mure Referee Pictutes. By -previous" 
-afarence pictore is mean, a reference I- or P-pictore ma, has alreedy been displayed 
and Is used tor motion compensation predion o, subsequen, plcdrrea ,ha,haveye, 
to bedisp.ayed.By Wreferenc8p | ctOTb ^ ap|cliiretha[[sio[)ed[sp|ayed 

■II U.J wnid , a Bin Been contalneu „, the Input srgnal bKsbeem and 

received before me current plctore to pen* ,he copying 0 , date bom 8. With ma 
modon vector, en Error Block o, only , he date, ifeny, .ha, cenno, be supplied by copying 
bom the reference pictures Is transmitted In pixel dote form. 
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Motion company vectors ,„ cunen, MPEG P. and 

This requires that the MPEG decod rs 
perform a heif-pe, on o, iuminance and chrominance vaiues from adjacent 

»e iuminance and chrominance vetoes fc, the ^ of ^ h ^ 

Pierre. Typtea, MPEG video decoders cany ou, «. ha,pe, upo „ „ 

^-^^-P^asm,^^,^^^ 
outeothuner. V*h stands resotuaon sterns, the ou*ut ma«* l00ks v* have the 

z rv ph9te as reference — *-* °° - - - — 

- ^ ~ — «■ <» *— . The resolution o.the 
^^P^^e^asmatofm.^^,^ 

, ' d69raded KPrMU *" — The additten of ha^ 

-ntetpoiaiion «o moaon compensation of video programs enhances me q uatny of the 
output when presented in the original resolution. 

Video presenter systems produce rectangute, lmages „ y 
»nes. ta top ta hotton, on a scree. The images are fcimed o, rectangutar a^ys of 
^rexan,,e.a, 72 0pi X e b pe r soa„r ra . w ^ 480sran ^ wp ^ reunder 
^e NTSO stendard fort he curren, resotuson standem used tn ,he United States and 
*P« and 578 scan iines „ ptehTO under the PAL standard for the cuneni resoiutfcn 
Randa n, used in Eurepe .Standard de»n M on pregrems are disptayed , ^ format8 . 
u_ 8,anu ™ J ' S5 otspiayuu 5 rSHf 55 pictllres 

^^^•^^^^"^.uccesa^^atepae,^ 
incudes the even iines o, a picture and a ho«om tteid « inoiudes the odd tines of a 
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plcture. Under NTSC. 60 fields per second are displayed. Under PAL, 50 frames p r 
second are displayed. 

Frequently it will be necessary to display a program that is broadcast or recorded 
under one standard, NTSC or PAL, on a system that is configured to display under the 
other standard. Such cases require frame rate conversion from one rate, 60 or 50 fields 
per second, to the other rate. Such conversions are from 6 to 5, or from 5 to 6, frames 
per second. The modes for such conversion are not specified by MPEG. 

In addition, many programs to be displayed on systems of either the NTSC or 
PAL standards are broadcast or recorded from motion picture film, in full frame images 
at rates of 24 or 20 frames per second. In such programs, these progressive images 
are recorded with all of the odd and even scan lines interleaved and encoded by frame. 
Such programs must undergo a frame rate conversion for display in 30 frames per 
second NTSC or 25 frames per second PAL frame rates. These conversions can be 
(1) from 24 frames per second to 25 frames (50 fields) per second, (3) from 20 frames 
15 per second to 30 frames (60 fields) per second or (4) from 20 frames per second to 25 
frames (50 fields) per second. This produces conversion ratios of from 4 to 5 frames 
per second . from 24 to 25 frames per second, from 2 to 3 frames per second, and again 
from 4 to 5 frames per second, respectively. 

Digital Video Disc (DVD) recordings include information in the bitstream Picture 
Headers that specify which frames are to be repeated to convert, for example, the 
24 fiames pei swuuilU of a motion picture record.ng to the 30 frames per second of 
NTSC video. Other programs such as Video Compact Disc (VCD) recordings do not 
specify which pictures are is to be repeated in a conversion, even though, to play such 
recordings on a PAL or NTSC system, such conversion must be conducted by the 
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receiver. Furthermore, when recordings are to be converted from PAL to NTSC, or 
NTSC to PAL. intelligence must be provided in the receiving system to define a repeat 
scheme that will effectively reproduce the program to the system video output standard. 

Furthermore, straight forward conversion systems have had certain minimum 
buffer memory requirements. In addition, the specified repeat order of DVD programs 
also requires a minimum amount of buffer memory to implement The conversion of 
24 frame per second progressive frame motion pictures to 30 frame per second NTSC 
video traditionally employs a conversion scheme referred to as 3-2 pull down, by which 
three fields are generated from two fields of a frame of the original picture by displaying 
one of the fields twice. In the case of a progressive frame encoding of a 24 frame per 
second motion picture film to NTSC 30 frame per second video, such 3-2.pull down may 
include, for example, displaying three fields from the two fields of one received frame 
and then two fields from the next frame, followed by three from the next then two from 
the next The sequence under MPEG is specified to be: top-bottom-top, then bottom- 
top, then bottom-top-bottom and then top-bottom from four consecutive frames of the 
original picture to produce five frames of display, that is, by displaying a ten field 
sequence out of every four frames of original data, for a 24 to 30 frame per second 
conversion ratio. 

Frame rate conversions, particularly those requiring the repetition of frames or 
fields, affects the design of the receiver. To repeat a field of a picture, either the 
««*uu«a Heia musi oe storea or the field or the same frame must be decoded more 
than once. This increases the decoder speed requirements, the required buffer 
memory, or both, affecting the cost and complexity of the receiver. 
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An additional problem presented bythe variety of conversion requirements is the 
complexity of the video decoding needed to dea, with the alternative conv rsion 
situations using methods of the prior art. 

Furthermore, not all video presentation systems are to be used in applications 
requiring the high resoiution and other capabilities of DVD or other MPEG compliant 
systems. Accordingly, in order to make the systems of differing performance 
capabilities available at optimum cost, video decoders and other system components 
shouid be capable of functioning in a variety of systems to provide a wide range of 
capabilities without imposing on all such systems the same memory and performance 
requirements. Since it is not economical to produce electronic circuits in small 
quantities for each application, prior art systems typically are not produced to 
economically serve each of the applications for which they are needed. 

For all of these purposes, the need to make frame rate conversions has the 
propensity to increase the complexity and cost of the video decoding system or the size 
and cost of the video buffer memory. 

There is a need, particularly for video presentation systems with standard 
resolution video programs, for efficient and reliable frame rate converaion to take place. 
Summary of the trjygjitifin 

• A primary objective of the present invention is to provide a video decoding 
system and method by which video programs can be efficiently and effectively 
conveneo rrom one frame display rate to a nother. It ,s a particular objective of the 
present invention to provide a video decoding system and method by which such frame 
display rate conversion can be made by repeating the display of fields from received 
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pictures to display a greater number of fields in a given presentation time interval than 
the number of frames in the interval that are received in the original program. 

A particular objective of the present invention is to provide an efficient and 
effective system and method for performing frame rate conversions such as, for 
5 example, 3-2 pull down conversions including pull down in VCD and DVD and NTSC- 
PAL or PAL-NTSC conversions. More particular objectives of the invention include 
providing for such 3-2 pull down while facilitating the use of commands such as pause, 
fast-forward, slow forward, reverse play and other such commands which are often 
referred to as "trick play" commands. Such objectives also include implementing frame 

10 skipping required in audio-visual synchronization. 

Another objective of the present invention is to provide in an MPEG video 
decoder one module and routine to handle frame rate conversions and other frame rate 
related issues, as well as frame rate related issues that are dependent on the amount 
of available buffer memory of the system in which the decoder is used. A further 

1 5 objective of the present invention is to provide an MPEG video decoder that performs 
a single decompression and transformation method regardless of the occurrence of 
frame rate conversion and the conversion rate and regardless of differences in the 
display sequences due to the frame rate conversion, if any, employed, or due to buffer 
memory size. 

20 A further objective of the present invention is to efficient and effective use of 

buffer memory and to facilitate the use of minimally sized buffer memory to buffer 
decoded video picture sequences for display during regular play, where frame rate 
conversions are required for program viewing, and during trick play modes and 
transitions into and out of trick play modes, particularly while maintaining optimal display 
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quality. An additional objective of the invention is to provide a memory management 
system operative to map decoded pictures to buffer memory and allocate buffer 
memory so as to allow for the sharing of memory locations by more than one field in a 
way that reduces memory requirements. 
5 According to the preferred embodiment of the present invention, an MPEG video 

decoder is provided with a decompression and transformation section which decodes 
a full frame of video on command by a single method that applies regardless of buffer 
memory and frame rate conversion considerations that would otherwise call for differing 
display sequences of the decoded pictures. The decoder is provided with a display 

10 control module that handles all frame rate and field sequence issues in response to 
host configuration information, particularly buffer memory size and system type (NTSC 
or PAL), and to host command signals, such as trick play commands, as well as to 
information in the received bitstream, particularly the sequence and picture headers and 
extensions. The display control module handles these issues, in a way that allows the 

15 other components of the decoder and of the display output logic to operate in a simple 
and consistent manner. 

In certain preferred embodiments of the invention, pictures are decoded in the 
order received and as buffer memory for the decoded pictures becomes available. The 
decoded pictures are assigned attributes that are stored in a table, with one attribute 

20 string associated with each decoded picture. Signals are sent to a field display logic 
section along with the memory address of the next field to be displayed along with the 
attributes needed for affecting proper display. These attributes designate which field 
of a picture is to be displayed (top or bottom), whether the memory is to be freed for use 
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by the decoder as the field is being read for transmission to the display, and whether 
the decoder is to be enabled to decode the next picture as the field is being display d. 

In the certain preferred embodiments, default attributes are predicted based on 
frame rate conversion considerations and then modified to give consideration to field 
5 display sequence information accompanying the pictures of the program. The 
generation of attribute tables considers buffer memory size, and field display order is 
modified to the extent necessary to allow the program to be reproduced at a proper 
display rate even when memory is small, using opposite field data where necessary. 
The field sequence order facilitates the use of output buffers for B-frame data that is in 

10 the range of from 0.53 to 0.67 frames in size. 

In various embodiments of the invention, use of available buffer memory is 
optimized by maintaining tables of offset variables and accessing a fixed table of 
memory pointers as fields of data are being displayed. The offeet data tables are 
identified to the display logic, which uses the data in the offset tables to indirectly 

1 5 address rows of memory in which the consecutive rows of field data for the field to be 
output have been stored by the decoder. The decoder loads offeet values Into the 
offeet tables as pictures are being decoded and rows of blocks of the picture are stored 
as memory becomes free. Preferably, two fell frame reference picture buffers are 
provided for storing two decoded reference I- or P-pictures and one 0.53 to 1.0 frame 

20 buffer is provided to buffer B-pictures, while four offset variable tables are provided, one 
to noid address offsets for both reference picture buffers and three to hold offsets for 
up to three different B-picture fields that can be each at least partially present in the 
output buffer at one time. 
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The invention provides versatility for various temporal up-sampling and down 
sampling schemes, particularly frame rate conversion schemes, and its operation is 
particularly smooth. The invention supports the use of less than three full frames of 
buffer memory, particularly that referred to as 2.53 mode or 2.53 frame DRAM memory 
configuration, as well as three frame and four or more frame video buffer memory. The 
invention also supports various trick play modes and their use simultaneously with 3-2 
pull down. The system and method provide versatile conversion and the ability to 
handle conversions between PAL and NTSC. with and without 3-2 pull down, in both 
VCD and DVD as well as other formats. 

These and other objectives and advantages of the present invention wDI be more 
readily apparent from the following detailed description of the preferred embodiments 
of the invention, in which: 
Prief Description of the n rawifm . 

Fig. 1 is a block diagram representing an MPEG receiver embodying principles 
of the present invention. 

Fig. 2 is a block diagram representing the ASIC portion of the receiver of Fig. 1. 

Figs. 3 is a detailed block diagram representing the DRAM buffer and video 
decoder portions of the MPEG receiver of Figs 1 and 2. 

Figs. 4A-4D are diagrams illustrating memory mapping alternatives forthe buffer 
memory of the receiver portions of Fig. 3. 

Fig. 5 is a timing diagram illustrating the writing and reading of a 0.533 frame 
output buffer similar to that of Fig. 4D for normal output. 

Fig. 5A is a timing diagram similar to Fig. 5 illustrating the writing and reading 
of a 0.533 frame output buffer with field repetition. 
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Figs. 6. 6A and 6B are flowcharts illustrating the operation of the RISC of the 
receiver of Figs. 1-3 in decoding pictures in a way that particularly facilitates frame rate 
conversions. 

Figs. 7 and 7A are flowcharts illustrating the operation of the RISC of the 
5 receiver of Figs. 1-3 in displaying fields in a way that particularly facilitates frame rate 
conversions. 

petted Description of the Prefer red Embodimen t* 

One embodiment of the present invention is diagrammatically illustrated in Fig. 1 
in the form of a digital television (DTV) audio and video presentation system 30. The 

10 system 30 includes a signal processor and controller unit 31 having a program signal 
Input 32 in the form of an antenna, a cable, CD ROM or other medium through which 
a digital input signal is received, a control input from a control input device 33 through 
which a user makes program and presentation format selections, a video output which 
connects to a video display or video presentation subsystem 34. and an audio output 

15 which connects to an audio amplifier and speaker system or audio presentation 
subsystem 35. 

The unit processor 31 includes a central processing unit or host CPU 36 which 
is programmed to process user commands from the control input device 33 and to 
operate a control system display 37, which displays information, menu selections and 
20 other information to the user an d which may or may not also function as an input 
device. The unit processor 31 also includes an Application Specific Integrated Circuit 
or ASIC 40, which, when provided with configuration and selection information by the 
host CPU 36, decodes the raw digital bitstream signal from signal input 32 for output 
to the video and audio presentation devices 34 and 35. The unit processor 31 further 
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Includes a local system clock 41, which connects preferably to the ASIC 40, and a 
buffer memory 42. The buffer memory 42 is In-line, sequential memory, such as 
dynamic random access memory or DRAM, which may be, for example, synchronized 
DRAM or SDRAM. The buffer memory 42 preferably a contiguous variable length audio 

5 buffer 44 for use by the ASIC 40 for audio signal processing and a video multiple picture 
buffer 48 for use by the ASIC 40 for decoded video processing. 

Fig. 2 diagrammatically illustrates the configuration of the ASIC 40. The ASIC 40 
is a single integrated circuit chip that is logically divided into a number of components 
or functions. The ASIC 40 includes a memory control and a data bus or gbus, 

10 represented diagrammatically by memory controller and bus 50, which has at least one 
received data input connection and a plurality of data connections to the ASIC 40 and 
the DRAM 42. The ASIC 40 has distributed among its various modules on-chip static 
random access memory or SRAM, illustrated diagrammatically at 51. The connections 
of the bus 50 include a connection host interface unit 52 which connects externally with 

15 the host CPU 36, and connections to the DRAM module 42 which Is external to the 
ASIC 40. 

The ASIC 40 also includes a demultiplexer or DMUX 53 which has an input 
connected to the signal input 32 of the unit processor 31 and an output connected to 
the received data input of the bus and memory controller 50. The Dh/lUX 53 also may 
20 have a text output connected to a teletex processor 54 that is also provided on the 
^ ASIC 40 for processing collateral information such as closed caption script, interactive 
menus and other text like data. The ASIC 40 further includes an audio decoder 55, a 
video decoder 56 and a local sub-picture generating unit 57. The audio decoder 55 has 
an input connected to one of the data connections of the bus 50 and an output 
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connected externally of the unit processor to audio presentation subsystem 35. The 
video decoder 56 has data connections to and from the bus 50 from which it received 
coded video data to process. The video decoder 56 decodes by decompressing, 
transforming and motion compensating the received video data, and sends decoded 
and processed video picture data through bus 50 to a video buffer 48 in the DRAM 42. 

The sub-picture generating unit 57 generates local picture information that 
includes control menus, display bar-graphs and other Indicia used in control Interaction 
with the user. A blender 58 is provided which combines the local video from the sub- 
picture unit 57 with teletex information from the telefax processor 54, and decoded 
video from the video buffer 48 of DRAM 42 that has been post-processed by a post- 
filter section 59oftheASIC40. The output ofthe blender 58 is connected externally 
of the unit processor 31 to the video presentation subsystem 34. 

The ASIC 40 is provided with a control bus 60 to which a control port of each of 
the components of the ASIC is connected to receive control signals in response to 
which they operate. The ASIC 40 includes a Reduced Instruction Set Controller or 
RISC 61 , which serves as the local CPU of the ASIC 40. The RISC 61 controls the 
functions ofthe components ofthe ASIC 40 by generating control signals and applying 
them through control data ports onto the control bus 60. The RISC 61 has a clock input 
that connects externally ofthe ASIC 40 to the local system clock 41, and has another 
input connected to phase locked loop c ircuitry or PLLs 62 within the ASIC 40 used to 
time Internal clock signals. — 

The information flow and memory configuration is illustrated in the simplified 
block diagram of Fig. 3, which shows the raw signal of the incoming bitstream at 
input 32 entering the DMUX 53 which extracts and sends certain headers to the 
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RISC 6, ,„„ routes dala mrough „, ^ ^ so (o an . nput ^ e3 ^ ^ 

ORAM 42. Preterabl, tee raw encoded date tern, »,e blbtrean , „ ^ buffered an 
■W*«««.«* B . h ». 1(llll4ta|(tth(||ai| 
■» M «he date H,^ the bus 50, separates vtdee. audio and sub-plcfcre date 
and teen wntes tels separated date Ihrougb tee bu8 „ and into ^ 

bUffer8: ^ F,F ° 65 ** ^ fram the host. „„. FIFO buffer 

66 for encoded audio data and one FIFO buffer 67 for encoded video data 

The FIFO buffers 63 are par, of the random access DRAM 48. bu, are caused 
^^nas^r^^rtnomemo^by^prog^^^^^^^ 
memory center 60 . ^ FIFO bulfera 63 have read and wrtte pointers whfcb 
America,* detenrtne «he addresses to and from whtch tee next „*, and read 
commandswrtteandread. ThaposH.onsofmesereadandwrttepotete.oanbsread 

R,SC61 "* - ' " "»0 61. As a result, the R,SC 61 oan move e 

-ad potnter of. for example, video F,FO 67 to repeat or sWp a picture, and oan 
suppress the advance o, tee wrtle pointer 1o pravem or cause a pfctere Ifom being 
overwnben to faclKate tee repeatetg SK^plng. or reordering o, pictores where desired 
This accommodates frame rate converaions and certain toe* ptey modes 

The vHeo decoder 56 lakes encoded video date one alloe a. a brae from tee 

re ad P oln te r^ lnvld eotepu,F,FObufter67andp ro cesses tee data macrobtoc* 
by macrobloc* In a decomprasslon and transformed seo«on 7,. whtoh exlracte 
■■..uunaaon rrom slice a nd picture headsrs tor interpretebon intematly by tee 
OacoderSS. Higher level headers, such as sequence headers and GOP headers which 
were extracted by tee DMUX 53 are grated by tee RISC 61 The teterpretebon of 
higher levet headers may involve storage of infomtetton to SRAM 51 on tee ASIC 40 
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or to a portion 72 of the DRAM 42 reserved for the storag of decoded header 
inflation, miscellaneous parameters and other such data. For example, information 
In the Video Sequence Header may contain picture size and aspect ratio data that might 
be used to by the ASIC 40 to reassign memory segment labels to the video buffer 48 
of DRAM 42 and to keep track of the sizes in the buffer memory 48 of the buffers 
needed for data buffering of reference pictures and the output picture. 

The processing of picture data by the video decoder 56 is performed on a slice 
or slice row basis in response to commands from slice decode software 88 in the 
RISC 61 which instructs the decoder 56 to read a slice or row of slices from the video 
FIFO 67, to decode the macroblocks of the slices so read, and to store the decoded row 
to DRAM 42 at an address provided by the RISC 61. These addresses may be to any 
one of the video buffers 73A. 73B or 78, depending whether the decoded video is of a 
reference I- or P-plcture or of a B-picture that is being processed for direct output to the 
dispiay 34. The slice decode software 88 is controlled by picture decode software 89 
of the RISC 61 to instruct the video decoder 56 to decode all of the rows of a picture. 

The decoding process controlled by one cycle of the picture decode software 89 
may include the decoding and storing of at most one fell frame of decoded / pictum 
frame data in either one of two full frame reference picture buffers 73A.73B of a video 
frame buffer portion 73 of the DRAM memory 42. Such /-p/cfon* are inverse DCT 
decoded and stored in the reference picture buffers without m otion compensation and 

the copying of data from previously stored reference pictures. 

The processing by the video decoder 56 includes the performance of an inverse 
DCT on the data by decode and decompression section 71 and the application of 
motion compensation calculations to the transformed data by a motion compensation 
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section 77 o,«he v*eo decker 56. The melon companion secaon 77 consbvcts 
P-P-cfun, frame data for storage in one of the referee pichae buflers 73A.73B by 
copying and retoca^ng pixel data tan, a buffered , or 5lora d ,„ ,„ e otoer one 

of toe outers 73A.73B. The P^res are produced by copytog dato from th e *a me 
pfototB In auch otoer buter aocording to motion vector data h toe received signal toa, 
Is read from the video FIFO buffer 67 by the video decoder 56. 

The video decoder 56 decodes and pen™ motion compensate, canons 
fortoeconstructenand butertng of B-ptotoreteas bycopyto gwl averaging dato *om 
sn f- pictore and a P picture, or b™ p^, ^ ^ „ ^ ^ ^ 

^73A.73 B .TheB^c ftre ssoconsbuc te da re «^„ asoneorhra(jek|sintoal) 
outou. outer 78 in DRAM 42. The oufcu. buter 78 is a one full frame pictore in sfce In 
some embodiment and in otoe, embodiments is less than one full frame, but a. a 
minimum is larger «han one half frame (one field) of a plcbae by an amount e,ual to a, 
-east one hoifconta, row of 8x8 video dafa blocks, and pmfembly by an amount e<,ua, 
to a. leas, one horizontal row of 18x16 video mactobtocKs. which la two rows of 8x8 
vldeoblocKs. Pictoredata Is^dbytoeoutouWngfleUpictoresseguenaaUynom 
toe outou, buffer 78. line by line, through the post ffitertng section 59 and to me video 
display 34. 

Both of toe video buffers 73 and 78 are divided Into h*, parts, one 73L.78L for 
storing pixel lumhanc data and one 73C.78C for storing pixel chmmtoance data. TWs 
» represented by toe .wo layem depicted h Fig. 3 tor .he outers 73 and 78. A master 
conbo. pmgmm In Ihe RISC 61 assigns Base_Addresses and Buffer Sizes to each of 
toe pens .L and -C of each of toe bute re 73A. 73B and 76, as we,, as a row index 
increment toa, ,s derived tern toe HortzrxitaLVW, of toe plctu™ of toe«e q „ence 
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The Base Addresses and Buffer__Sizes can be changed as picture format changes in 
the received program. Further, since frame sizes can vary from program to program, 
a host DRAM configuration 42 that constitutes a 2.5+ buffer for one program can be a 
three frame buffer for another program. For example, a 2.53 buffer for an MPEG-2 
5 program is more than adequate to store three frames of an MPEG-1 program. Lower 
cost host systems in which the ASIC 30 can be used may be provided with a minimum 
DRAM 42 necessary to provide at least a 2.533 frame for the highest definition or 
largest frame size that the system is expected to present. Other higher performance 
systems will provide larger buffer memories of, for example, five times the largest frame 

10 size expected for optimal performance. With larger buffer memories, the control 
program of the RISC 61 continuously reassigns the Base_Address of the next buffer 
for the optimum use of available memory and for optimal preservation of data to meet 
the highest performance requirements. Such a higher performance system is explained 
This is further explained in related applications Serial No. 09/178.803. Management of 

15 Trick Playback of Digital Video Data and Serial No. 09/ . . entitled Trick Playback 
of Digital Video Data referred to above. 

According to the preferred embodiment of the invention, the RISC 61 includes 
field sequence control logic 80 to control the order of field data to the buffer 78. 
Including the performance of frame rate conversions. Conversions that are made 

20 include, for example, 3-2 pull down conversions from 20 or 24 frame per second VCD 
or DVD to 25 frame per second PAL or 30 frame per second NTSC. as well as NTSC- 
to-PAL and PAL-to-NTSC conversions. The conversions are implemented while 
facilitating the use of commands such as pause, or may be suspended during other 
commands such as fast-forward, slow forward, reverse play and other trick play modes. 
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in addition, the field sequence cento, logic 80 includes field r pea, nvVne logic ft,, 
determining the repeating cf fields to be done when tame rate conversions so require, 
end for Implement frame skipping in audiovisual synchronization end In other 
situations where frame skipping Is required. 

This field sequence control logic 80 is contained In a single module within the 
RISC 61 which, atong wfth a field sequence attrtbute seMng submuftte 06, includes the 
enure routine needed to handle freme rate conversions and omer tame rate related 
issues. ,„ add«on, meconbo, log* 80 confrols outer .tame raferefcfed Issuesfhafare 
dependent on me amount of buffer memory provl ded by fhe many various forms of 
systems 30 In which me ASIC 40 is used. The confro, logic 80 confro* these Issues 
by determining the appropriate Held display sequence, synchronizing the opemflon of 
the decoder 56. Insfrucbng the DMUX 68 » necessary, and mappfcg end fimlng ,he 
storing of decoded video slice data to the ou*u, buffer 78 to cany ou, the fieid display 
sequence ma. me confrol logic 80 determines is appropnate. The con.ro, teglc 80 
allows a single decompression and tomsformaton memod regardless of ma occurrence 
of ftame rate convent and regardless of the detetreined field dfeplay sequence or me 
inclusion .herein of fieW repetibon or field skipping. accommoda.es outpu. field 
buffers 76 ma, are one frame In size and buner, ma. ere smaller, such as ou*u, 
.buffers 78 ma, arsons one oratew block-rews German one half frame (one field). 
in g e neral, when video date of a bksfream Is receive d by me DMUX 53. Vkfeo 
. 1 UJ,, »„ u ,mai,on is exfracted contalnmg plctere stze and aspect ralio date 
bl, rate fimite and omer globe, parameters, video Sequence Extensions are also 
included in some sequences which may also be conteln Cher Infomtabon applicable to 
all plcteres of Ore sequence such as a Progressiva Sequence bl, which Indicates ma. 
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the sequence contains only Progressive Frame pictures, a Chrominance Format code, 
original video format (e.g., NTSC. PAL. other) and other variables. In MPEG-2, such 
parameters can be changed in the course of a program by information in the Sequence 
Header or Sequence Extension. This header and extension information is 
communicated to the RISC 61 for use in processing and displaying the pictures of the 
sequence. 

In the same manner GOP Headers and, in MPEG-2 GOP Extensions, are also 
received by the DMUX 53 and communicated to the RISC 61. In the video decoding 
and presentation process, as groups-of-pictures (GOPs) of each sequence are 
received, presentation delay requirements and other data relevant to the picture and 
all of the P- and B-pictures in the entire GOP are extracted from the GOP Header and 
interpreted by the RISC 61, which Instructs the video decoder and other ASIC 
components. 

• As each picture of the GOP is received, a Picture Header is extracted and 
interpreted by the RISC 61 to determine the picture type (e.g., /, P or B), the display 
order of the picture relative to other pictures or its relative place in the stream, and 
relevant other information concerning the picture. The Picture Header in MPEG-2 is 
accompanied by a Picture Extension that includes information on picture structure 
including whether the picture is a frame DCT picture or a field DCT picture, and if field 
DCT, whether of a top or bottom field and bits indicating whether to display top or 
bottom field first, whether to repeat the first field, as where frame rate conversion is 
required, and other variables. In response to information in these headers, the RISC 61 
controls the video decoder 56. particularly the decompression and transformation 



WO 00/59219 

PCT/USOO/01023 

*om the picture in one of bu)fere ^ ^ „ 7aot0lemm42 
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mapping and ^ fc u s pa(em 
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.con^^^noe^. A slice decoding software aeclion 88 of the RISC 61 

« en*, picu.re row C ^ data . ,„ read|na ^ ^ . ^ ^ ^ ^ 
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!^^^~^ P*» da* a, e «me. For frame encoded 
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cosine transform decoding (IDCT) and buffering, in a reference picture buffer 73 of the 
video buffer 48, a full frame reference pictures, either in the form of an /-picture, which 
is stored without reference to data from previously received pictures, or of a P-picture, 
which is stored after copying data from a previously received /- or P-picture. Such a 
5 reference picture is preferably stored as a full-frame picture containing the alternately 
even and odd scan lines of the frame in the top to bottom order in which they will 
appear in the displayed picture. 

In the preferred embodiment, two portions of the buffer 73, portions 73A 
and 73B, store the previous full frame picture in the form of an / or a P-frame and the 

10 next full frame picture in the form of a P frame. Such P-frames, are written to this 
memory 73 by copying, with or without motion compensation, portions of the previous 
picture from the other portion 73A.73B of the buffer 73. When a further P-frame is 
received, it overwrites one of the frames in the memory portions 73A or 73B, typically 
the earlier of the two in presentation order, to become the next, "next" frame, while the 

15 frame in the other portion 73A or 73B of the buffer becomes the next "previous" frame. 
When a further /-frame is received, it will overwrite one of the frames in the memory 
* portions 73A or 73B. Storage of a frame picture in either of the buffer memory portions 
73A and 73B involves storage into halves 73L and 73C of such buffer portion the 
respective luminance and chrominance data for the picture being stored. 

20 When an /-picture is received, it is stored in one of the full frame reference 

picture buffers 73. An /-picture is also read from its reference picture buffer 73 to the 
post filtering section 59 of display hardware 87 when it is to be displayed. When a P- 
picture is decoded, extrapolation of the data from a stored /- or P-picture is carried out 
for each macroblock in accordance with a motion included in the macroblock data, and 
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the P-plcture is stored In one of the full frame reference picture buffers 73. When a 
e-picture is received, interpolation of the data from two stored reference pictures is 
carried out by copying data in accordance with motion vector data from the 
macroblocks. fl-picture data is decoded in presentation time order and is according 
written directly to the output buffer 78. In the case of P-pictures. the fuii frame P-picture 
is constructed and stored in one of the portions 73A.73B of the buffer 73 from data 
copied in part from the previous /- or P-picture in the other portion 73B.73A of the buffer 
73. The motion compensation processing is handled by a motion compensation 
portion 77 of the hardware of the video decoder 56. 

The parsing of data to the reference picture buffers 73 may be carried out in a 
straight-forward manner, utilizing the base address of the respective buffer and a row 
index corresponding to the vertical position of the slice in the picture, or it may be 
carried out dynamicaliy. which is the way in which slice level parsing of the output 
buffer 78 is achieved. The output buffer 78 has a size within a range, preferably, of 
from one row of blocks or macroblocks more than one half of a frame up to a full frame. 
Slice level parsing is achieved by dynamically allocating empty rows in buffer 78 of the 
DRAM 42. When the buffer 78 is less than a fell frame in size, the parsing of siices 
when stored to the buffer 78 as well as the decoding of pictures and the outputing of 
lines to the display must be particularly controlled. 

The RISC 61 has a section 79. which is connects with the host interface 52 and 
cock „ or the ASIC 40. to control the di splay mode and timing and trick play 
operations such as fast and slow forward, pause and reverse. This trick play and 
display mode section 79 receives timing information from the system field clock 41 and 
user control signals from the host control input 33. The section 79 sends commands 
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to a picture decode control 81 and to a field display control 82. It informs the field 
display control 82 when a buffered decoded picture is to be output and tells the decode 
control 82 which pictures to next decode and write to the buffers. It also signals the 
decode control 81 and also directly controls the DMUX 53 to implement the picture 
sequences necessary to produce trick play modes such as fast and slow forward, 
reverse, pause, skip, etc. 

The decode control 81 controls the order in which picture data is decoded and 
ultimately sent to the output buffer 78 as well as the reference, picture buffers 73. The 
decode control 81 controls the decoding process by the video decoder 56 and those 
operations of the DMUX 53 that relate to the timing of picture decoding and decoding 
order. The picture decode control 81 includes the field sequence control 80 and the 
field repeat subroutine thereof 96, the picture decode software 89 and the slice decode 
software 88. It is a primary function of the picture decode control 81 to insure that 
pictures are decoded and the buffers 73 and 78 are loaded with decoded picture data 
15 in the order and by the times they are needed for reference or for display. 

The display control 82 controls the outputing of decoded data to the display 34. 
The display control 82 includes a display time command routine 95 which responds to 
a field output timing signal from the display mode software 79 and attribute data from 
field sequence software 80 data to command the output of a specific field of a picture 
20 to the display. The command routine 95 communicates with address hardware and 
software 86 which determines the addresses of the buffer 42 from which slices of the ' 
buffered pictures are to be output and consecutively processes those slices to send 
them to the display. A memory management data section 83 of slice storage address 
data informs the address software 86 of the buffer locations from which the slices to be 
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displayed are located. The memory management data section 83 Is logically part of the 
display control 82 but may include tables that are physically stored in the buffer 
memory42. The field dteplay control 82 informs the decode control 81 ofthestorage 
locations that are freed by the output of slices of picture field data, while the decode 
control 81 keeps supplying decoded data of the next future pictures for display to the 
buffer memory 42 while informing the dispiay control 82, by ioading tables in the 
memory management data section 83, of the locations of the slices as they are 
decoded. The display control 82 reads slices of data, line by line, from field pictures 
stored in the picture buffers of DRAM 42 as the display is due. while the decode 
control 81 decodes the next pictures to be displayed and parses the decoded slices to 
the buffers as buffer space becomes available. 

Preferably, the slice level parsing is achieved by allocating the first two available 
rows in the table for the storage of a slice that has been decoded by the decoder 56, 
since the motion compensation section 77 decodes 16-line macroblocks. The 
availability of the rows of the buffer 78 is maintained in a memory-empty flag table 84 
of the memory management data section 83. The memory-empty flag field is made up 
of three 32-bit words, which is more than adequate to contain one Row.Empty bit for 
each of a maximum of 72 8-line rows of a PAL-format picture. Each one of the 
Row.Empty bits of the flag field or table 84 corresponds to one physical row of the 
buffer 78. Each such row has an offset add ress, relative to the Base.Address of the 
UU " Br '°* re P rese «' l « d bv ■ value in an address pointer table 85, having one row 
corresponding to each bit of the flag field table 84 and having two columns, one 
containing the offset address of the row in the luminance part of the buffer 78 and one 
containing the offset address of the row in the chrominance part of the buffer 78. The 
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"emory addrassea in fte ftfter 78 a, which fte finance daft and fte chrominance 
daft of fte decoded stfce ara slorad. The size offte fthft b 5ufliclent to ^ ^ 
for fte maximum 36 etice tocabons per flaid. Tahta 91 sftras .he Reasons of fte fts, 
decoded Maid, while 92 and 93 sftra iocabons ftr ,he second decoded B-fietd. „ fta 
B picture is a frame picftra. ft bl8 91 storaa fte top ne,d, ftr exampte. wh»e ftftes 92 
and 93 store fte hottom neid. Two fthlas ara us ed for fte bottom feM „, ^ , na 
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may be decoded. Vtften decoded raws o, btocto ara sftrad. one or .wo ftcatlons ara 
wn«en to a ftbta. ft fta case of a fl «d ploftra. two ftcaUons are wmftn ft fte same 
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table. In the case of a frame picture, one location is written to one table for the top field 
and one is written to one table for the bottom field. 

In addition to tables 91-93, one fixed table 94 is provided for both the reference 
picture buffers 73. This table is set up when the buffers are configured and makes the 
addressing scheme for the reference pictures independent of the picture type. When 
the RISC 61 determines that a particular field is to be displayed, a Display_Field 
command is sent from the display mode routine 79 to display time command routine 95 
with information for use by the memory allocation routine 86 and the display 
hardware 87, identifying the table 91-94 to be used for retrieval of the field data to be 
output. The memory allocation routine 86 retrieves the row locations from the 
designated one of the tables 91-94, retrieves the rows of the field in order from the 
addresses in the address table 85 Identified by the location data from the table, and 
then sends the lines of the rows of retrieved blocks of the field to the display. 

In preferred embodiments of the invention, the field display sequence 
control logic 80 in the RISC 61 synchronizes and controls the operation of decoder 56 
and the control of the output buffer 78 (via the memory controller 50) to handle the 
sequencing of the outputting of fields from the buffer 78 to the display 54 as well as the 
upstream operations of the decoder 56 and memory controller 78 in decoding and 
storing the field data to the buffer 78. In certain embodiments of the invention, the 
sequence control logic 80 also has the capability of controlling the DMUX 53 and 
decoder 56 so as to time the reading of pictures from the FIFO video buffer 67 to the 
video decoder 56 and signal the DMUX 53 to allow or suppress the overwriting of the 
FIFO buffer 69 with new picture data, to support the multiple decoding or the skipping 
of video pictures. 
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The sequence control logic 80 determines the need for frame rate conversion 
and controls these functions so as to regulate the adding or skipping of fields or frames 
to bring about any frame rate conversion that is required to match the input video frame 
rate to that of the display 34. The sequence control logic 80 regulates the display 
sequence of the fields in response to information in the picture data bitstream and in 
accordance with configuration information and commands from the host The 
information from the host to which the control logic 80 includes configuration data 
relating to the size of the output buffer 78, information regarding the format of the 
receiver (e.g., NTSC, PAL, etc.) and commands such as trick play mode commands 
10 (e.g., pause, fast-forward, reverse, etc.). 

The sequence control 80 determines the order in which fields are to be decoded 
and when and to where In the buffer memory 78 decoded slices are to be written. It 
controls when the video decoder 56 decodes these slices and tells the decoder 56 to 
which rows of blocks of the output buffer 78 the decoded slices are to be written. The 
control logic 80 stores information regarding the locations in the buffer memory 78 of 
the various rows of data from the various fields of the input pictures in tables 82 in the 
buffer memory 42. From information in the video bitstreams, or where the bitstream 
does not contain the information by making certain default assumptions, it determines 
the display sequence of the fields of the received pictures. Where the frame display 
rate required of the display 34 differs from tha t at which the original received program 
was recorded, the control logic 80 specifies which fields are either repeated or skipped 
to make the display of the picture sequence acceptable, and where possible. MPEG 
compliant. 
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Where the video decoder is fast enough and the output buffer is large enough, 
most display sequence conversions are straight forward. However, decoder speed and 
memory may be limited due to cost or other factors by host specifications. The field 
sequence control 80 facilitates the satisfactory use of lower speed decoders and, more 
particularly, of a host DRAM memory 42 that is relatively small in size. In certain 
embodiments, a full frame output buffer 78 provides for the necessary field repeatition. 
In other embodiments, a more economically sized DRAM memory of about .53 to .60 
frame is accommodated by repeating fields in a less optimum sequence. In still further 
embodiments, the use of limited DRAM memory is facilitated by decoding certain fields 
more than once from the input FIFO buffer 63, with the sequence control logic 80 
controlling the DMUX 53 to prevent overwriting in the video FIFO 67 of frames that are 
to repeated while triggering the video decoder 56 on a slice-by-slice basis enable field 
repetition by rotating use of limited rows of the output buffer 78. The manner in which 
the field sequence control 80 accomplishes its objectives and performs its functions can 
15 be best understood from the examples below. 

Full Framt* O utput Buffer 

In the following set of examples, the DRAM buffer includes an output buffer 78 
that is capable of storing a full frame output picture. The display 34, for example, may 
be configured to display a color image of 480 scan lines at, for example, 720 pixels per 
line, with lines interleaved from top and bottom fields. In an NTSC system, such fields 
are aispiayed at intervals of l/60th second apart to output a full frame of the entire 
480 odd (bottom field) and even (top field) lines in 1/30th second. The configuration of 
DRAM 42 in which the output buffer 78 is a single full-frame output buffer is referred to 
as a 3-frame video buffer configuration because it includes two full-frame reference 
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picture buffers 73A and 73B plus the full-frame output buffer 78. A full-frame output 
buffer 78 includes. In these examples, storage for 480 scan lines or 60 8-line blocks or 
30 16-line macroblocks. Data storage arrangements for such a full-frame output 
buffer 48 are illustrated in Figs. 4A, 4B and 4C. 

One configuration for storing decoded video data in the a full-frame buffer is in 

16-row macroblocks with the rows stored in order, 0,1 15, the rows of the top and 

bottom fields being interleaved, as illustrated in Fig. 4A. Such storage is more suitable 
for storage of reference pictures, which are sometimes used on a frame basis, than for 
pixtures being output for display, which in entertainment systems are almost always 
displayed on a field basis, one field followed by another. In Fig. 4B, the output 
buffer 78 is illustrated with a full frame of video picture data stored with the 8 lines of the 

top fields 7, 7*, stored in adjacent rows in one half of the buffer 78 with the 8 lines 

of the bottom fields B 1t ...B M stored in adjacent rows in the other half of the buffer 78. 
In Fig. 4C, the output buffer 78 is illustrated with a full frame of video picture data stored 

with the 8 lines of the top fields 7, 7 30 and the 8 lines of the bottom fields B f ,...B 30 of 

each of the 30 rows of macroblocks stored in separately, but with the field data of the 
macroblocks adjacent. 

Where the pictures are DCT encoded by field, the field pictures are sequentially 
DCT decoded and written to the output buffer 78. With such field pictures, the output 
buffer configuration of Fig. 4B is convenient, as one fi eld can be decoded and written 
to the one half of the buffer while a previously decoded field is being read from the 
buffer 78 to the display 34. With frame DCT encoded pictures, the top and bottom field 
lines of each slice are decoded together, so that the storage scheme of Fig. 4C is likely 
to be more convenient. With frame encoded pictures, the top and bottom fields of the 
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decode the next frame. 
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are successively sen. to the display. Assuming frame dCT encoded pictures of which 
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thedispiay. The rows of memory in which slices V,.V, and T, have been outou. and 
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*ame V, end V, . while me blocks 8- g m of me bottom field of me fire, frame 

remain in me memory. This allows me decoder 56 „ process me next „, „ ^ 
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required due to a frame rate in the incoming data tha, is lower man ,ha. of me 
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display 34. If the first field of a frame that is to be output (e.g., the top field) is to be 
repeated after the other field (e.g.. the bottom field) of the frame is output, the display 
sequence control signals the decoder 56 to suspend decoding of the next frame until 
the second field is being displayed, and to delay the overwriting of the first field until the 
first field has been repeated. While all of the mapping techniques of Figs. 4A-4D are 
useable, the technique of Fig. 4D is used in the examples, which relate to programs 
that include DCT frame encoded pictures. 

Example 1 

The case where no frame rate conversion is required and a 3-frame video buffer 
is available includes the situation where pictures in the program have a display rate that 
is the same as that of the display 34. as, for example, a program of 30 frames per 
second being presented by a system having an NTSC display format The output 
sequence of the fields of the successive frames of the picture, which are identified by 
the superscripts 1, 2, ... etc., is either 

TB\VB 2 ,1*B 3 ,...,ete. 
or fl , r.B 2 7 2 .B 3 T 9 ....efc., 

depending on which field of a frame is to displayed first. The solution is straight 
forward. 

Example 2 

The case where 3-2 pull-down conversion is required with a 3-frame video buffer 
available includes the situation where, for example, a motion picture is encoded 
progressive frames at 24 frames per second to be presenteci by a system having 
NTSC display format of 30 frames (60 fields) per second. This requires a 4:5 frame 
rate conversion. In this case, every four frames of a program bitstream in the form of: 
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VB\ VB 3 , VB 3 , VB* 

must be output as five frames. MPEG specifies that such conversion involve the 
outputing the first field of the first frame Wee as the first and third fields and the 
outputing of the second field of the third frame as the sixth and eighth fields of a ten 
field sequence. Fields of any given frame are output in the order TB, BT. TBTorBTB. 
For 4:5 frame rate conversion, the output sequence of the fields of the successive 
frames 1, 2, ... etc. of the picture is: 

VB 1 t VB*,T l B 3 t T i B 3 ,VB 4 
A similar result can be achieved with the sequence: 

B i r,B i r i ,B*r,B 3 T i ,B 4 r 

To output the sequence W.rflW.W.W. the display sequence 
control 80 controls the decoding and output as follows: 

Start with decoding of the first frame and storing V and & to output buffer; then 

(1) Output V, surpress decoding of the second frame; 

(2) Output B\ allow decoding of the second frame with overwrite of & only as 
memory is freed; 

(3) Output V again while allowing decoding of the second frame to continue 
overwriting V as memory is freed; 

(4) Output B 2 , allow decoding of the third frame with overwrite of & only as 
memory is freed; 

(5) Output 7* while allowing decoding of the third frame to continue overwriting 
T 2 as memory is freed; 

(6) Output B 3 , surpress decoding of the fourth frame; 
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(7) Output 7* allow decoding of the fourth framed overwrite oftf only as 
memory is freed; 

(8) Output B 3 again while allowing decoding of the fourth frame to continue 
overwriting B 4 as memory is freed; 

(9) Output r while allowing decoding of the fifth frame overwriting V as 
memory is freed; 

(10) Output Awhile ailowing decoding of the fifth frame to continue overwriting 
B* as memory is freed. 

etc. 

Example 3 

The case where 3-2 pull-down conversion is required with a 3-frame video buffer 
available in the case of, for example, a motion picture is encoded in progressive frames 
at 20 frames per second to be presented by a system having an NTSC display format 
of30frames(60fields)persecond. This requires a 2:3 frame rate conversion. Inthis 
case, every two frames of a program bitstream in the form of: 

TBWB 2 

must be output as three frames. Such conversion may involve the outputing the first 
field of the first frame twice as the first and third fields and the outputing of the second 
field of the second frame twice as the fourth and sixth fieids of a six field sequence. 
The output sequence of the fields of the successive frames 1, 2, ... etc. of the picture 



VB^PB'.PB 2 
A similar result can be achieved with the sequence: 

B'r.B'V.&T* ' 
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To output the sequence TB\ V&, ^B 2 , the display sequence control 80 controls 
the decoding and output as follows: 

Start with decoding of the first frame and storing V and B 1 to output buffer; then 

(1 ) Output V, surpress decoding of the second frame; 

(2) Output B\ allow decoding of the second frame with overwrite of B 1 only as 
memory is freed; 

(3) Output V again while allowing decoding of the second frame to continue 
overwriting V as memory Is freed; 

(4) Output B 2 , surpress decoding of the third frame; 

(7) Output T 2 , allow decoding of the third frame with overwrite of B 3 only as 
memory is freed; 

(8) Output B* again while allowing decoding of the third frame to continue 
overwriting B 2 as memory is freed. 

etc. 

Example 4 

The case where conversion is required with a 3-frame video buffer available 
includes the situation where, for example, a PAL program frame encoded at 25 frames 
per second is to be presented by a system having an NTSC display format of 30 frames 
(60 fields) per second. This requires a 5:6 frame rate conversion. In this case, every 
five frames of a program bitstream in the form of. 

VB\VB 2 ,r > B 3 ,rB\rB s 

must be output as six frames. Such a conversion may involve the outputing the first 
field of the first frame twice as the first and third fields and the outputing of the second 
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•eld of me W fra™ as the si*h ana eight* feIds 0 , a ^ ^ ^ 
output sequence of the fields of the successes frames 2. ... 8to . of plctun> „ : 

TB'.r&.T'e'.T'B-.rB-.np 

A similar result can be achieved with the sequence: 

B'r,B'T°,&P,B , r,B-r,B>7< 

To output the sequence VB'.TV. V*.V».rm. the display sequence 
control 80 controls the decoding end output as follows: 

S b dwifi,deood^offi W fi^ (ianwandsWrtngr ^ e , too ^ b ^ t(ien 

(1) Output T. surpress decoding of the second frame: 

(2) Output a', allow d9codlna of the second frame wfrh overwrft, of e' only as 
memory is freed; 

(3) omtu, V again white allowing decoding of the second frame to continue 
overwriting V as memory is freed; 

(4) Output sr. allow decod( „ fl „ „, , hw ^ ^ Meiw)(s ^ ^ ^ ^ 

memory is freed; 

(5) Output 7*while allowing decoding of <he Md frame to ovenw|tjng 
T 2 as memory is freed; 

(6) Output B 3 , surpress decoding of the fourth frame; 

(7) Output 7* allow decoding of the fourth frame with overwrite of 0> oniy as 
memory Is freed; 

<8) * Wh " e aHowin 9 d ^ding of the fourth frame to continue 

overwriting B* as memory is freed; 

(9) Output V while allowing decoding of the fifth frame overwriting V as 
memory is freed; 
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(10) Output & again while allowing decoding of the fifth frame to continue 
overwriting B* as memory is freed; 

(11) Output 7 s while allowing decoding of the sixth frame overwriting 7 s as 
memory is freed; 

(10) Output S 5 while allowing decoding of the sixth frame to continue 
overwriting 8 s as memory is freed. 

etc. 

Example 5 

The case where 3-2 pull-down conversion is required with a 3-frame video buffer 
10 available includes the situation where, for example, a motion picture is encoded in 
progressive frames at 24 frames per second to be presented by a system having an 
MTSC display format of 25 frames (50 fields) per second. This requires a 24:25 frame 
rate conversion. In this case, every twenty-four frames of a program bitstream in the 
form of: 

15 TB 1 ,VBi l ,r&.V& t ...,T i <B 34 

must be output as twenty-five frames. Such conversion involves the outputing the first 
field of the first frame twice as the first and third fields and the outputing of the second 
field of the frame 3+N as field 6+N and 8+N of a 25 field sequence, where N may be 
any number less than 22, but is preferably zero. The output sequence of the fields of 

20 the successive frames 1 , 2, ... efc. of the picture is: 

rB 1 ,VB 2 ,T 2 B 3 ,T'B 4 ,rB* T"B 24 

Otherwise, the display sequence control 80 controls the decoding and output similar to 
the method of example 3. 
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Example 6 

The case where 3-2 pull-down conversion is required with a 3-frame video buffer 
available in the case of, for example, a motion picture is encoded in progressive frames 
at 20 frames per second to be presented by a system having an PAL display format of 
25 frames (50 fields) per second. This requires a 4:5 frame rate conversion as 
described in example 3. 

Example 7 

In cases where conversion is from NTSC to PAL, for example, a 6:5 conversion 
rate is required, calling for the skipping of one frame, or two fields, for every six frames 
of input. If the two fields skipped are not from consecutive frames, the reversal of the 
display order of the fields of intervening frames will be required. This is carried out by 
signals from the display sequence control 80 to the video decoder 56. 
Half*- Frame Outpn^ SuSSC 

In the following set of examples, the DRAM buffer includes an output buffer 78 
that is capable of storing one half of a frame, or one full field, plus workspace of one or 
more rows of blocks, but less than a full frame of the output picture. Ifthe display 34 
is configured to display a color image of 480 scan lines, the output buffer will hold at 
least 248 scan lines. In preferred embodiments the output buffer has a capacity to 
store 256, 264, 272 or 288 scan lines of the output picture. The configuration of 
DRAM 42 in which the output buffer 78 holds one field plus a few rows of blocks is 
referred to as a 2.5x-frame video buffer configuration because K includes two full-frame 
reference picture buffers 73A and 73B plus an output buffer 78 that holds 0.5 frames 
plus the additional row(s) of blocks. 
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Fig. 5 represents a memory map and timing diagram showing how the display 
sequence control 80 regulates the storage and output of the top then the bottom fi Id 
of a first frame followed by the top field of a second frame (i.e., VB\ T 2 ...), where the 
output buffer has capacity to store 0.533 frames, which is one row of 16x16 
5 macroblocks (two rows of 8x8 blocks) more than one field (one-half frame) of output 
picture. This is referred to as a 2.53 frame video buffer. The 0.533 frame output 
buffer 78 includes storage for 264 scan lines or 32 8-line blocks or 16 16-line 
macroblocks. Sixteen macroblock sized rows of memoiy are identified by the numbers 
0 15 a,on 9 the left side of Fig. 5. The initial arrangement of the rows of data has the 

10 eight lines of the top field of a frame stored in a memory row adjacent the eight rows of 
the bottom field of the same frame, similar to the arrangement of Fig. 4C. This 
arrangement is short lived as the display sequence controller 80 and memory 
controller 50 write rows of blocks Into whatever rows become available first by the 
readout of data that is no longer needed. 

15 Example 8 

The case where no frame rate conversion is required and a 2.53-frame video 
buffer is used is illustrated in Fig. 5 for a program of 30 frames per second being 
presented by a system having an NTSC display format which displays frames at the 
same 30 frame per second rate. Time intervals of 1/60th frame, or 1/1 800th second, 

20 are listed along the top in Fig. 5. The output sequence of the fields of the successive 
frames of the picture, which are identified by the superscripts 1, 2, ... etc.. is illustrated 
as: 

rB\1*B 2 ,T i B 3 ,...,etc. 
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With the 2.53 frame buffer, revolving use of the memory is required and is provided by 
initially writing the top half of the first frame, or 16 slices of top and bottom fields of the 
frame, to the memory rows as indicated by the notations T 1 B t 1 ,T 2 B 1 2 ,T 9 3 B i 3 ,...,T t ie B 1 16 
at the left in Fig. 5. The superscript designates the first frame while the subscripts 

5 designate the row of macrocodes of the picture. While these data are shown as being 
present in the memory at time interval 0, it is only necessary that each row of blocks be 
written to the buffer before that row is to be sent to the display. Time interval 1 shows 
the top field of the top row of macroblocks V 1t represented by the circle, being sent to 
the display. Time interval 2 shows the top field of the second row of macroblocks T 2 

10 being sent to the display. As a result of the output of the top fields of rows 1 and 2, the 
rows of memory that they occupied are available for the writing thereto of the 17th row 
of blocks of the top and bottom fields of the frame, fields T 1 17 and B 1 17$ of the frame, as 
indicated by the squares in the figure. These rows are the first and third block rows of 
the memory, or the top halves of the first and second macrobiock rows, rows 0, and 1 . 

15 As shown, the first frame is fully decoded and written to the buffer 78 by time interval 
29, which is necessary since the last row of macroblocks of the top field is sent to the 
display in time interval 30. 

In time interval 31 , the output of the first row of the bottom field B\ begins. In 
interval 32, bottom field row B\ is output By time interval 33, as a result of the output 

20 of the bottom fields of rows 1 and 2, the rows of memory that they occupied are 
available for the writing thereto of the first row of blocks of each of the top and bottom 
fields of the next frame, frame 2, which are fields T 2 1 and B 2 „ of the frame, as indicated 
by the squares in the figure. These rows are the second and fourth block rows of the 
memory, or the bottom halves of the first and second macrobiock rows, rows 0, and 1 . 
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Regardless of the decoding of the second frame, the last row of macroblocks of the 
bottom field is sent to the display in time interval 60. 

During intervals 67-90, the top field of the second frame is output in the same 
manner that the top field of the first frame was output during intervals 1-30. It should 
be noted that at interval 61 , the entire bottom field of the first frame. B%, is still in the 
output buffer memory. The top field of the first frame, T„ has, however, been 
overwritten. Therefore, if it had been necessary to repeat a field of the first frame, the 
data from the top field of the first frame, V v cannot be repeated to the top field portion 
of the display, but only the data from the bottom field, B\, unless it were possible to 
10 decode the first frame again. This example assumes that the video decoder 56 cannot 
be operated to decode frames twice in the time available. The way that the display 
sequence control 80 handles this situation Is set forth in the next example. 

Example 9 

The case where 3-2 pull-down conversion is required with a 2.53-frame video 
15 buffer covers the situation where progressive frames at 24 frames per second must be 
presented by an NTSC display at 30 frames (60 fields) per second, requiring a 4:5 
frame rate conversion involving the repeating of one field in every four received. In this 
case, every four frames of a program bitstream in the form of: 

r8',T*B s ,T*B 3 ,rB 4 

20 must be output as five frames. As stated above, it is not possible to repeat data fr om 
the top field of the first frame, V v since it is overwritten by the time the bottom field, fl'„ 
is output. The display sequence control 80 handles this by repeating the bottom field 
of the first frame, B%, in place of the top field, which accommodates the case where the 
decoder 56 does not have the time to decode the first frame again. This is illustrated 
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in Fig. SA. For the 4:5 frame rate conversion with a 2.53-frame video buffer, the output 
sequence of the fields of the successive frames 1, 2, ... ete. of the picture is: 

A similar result can be achieved, where the bottom field is to be written first, with the 
sequence: 

B'v,rr,B i B 3 ,T > r>,B t p 

To output the sequence VB'.B'B'.VB'.rr.rB' with a 2.53-frame video buffer, 
the display sequence control 80 controls the decoding and output as follows: 

Start with decoding of the first frame and storing V and ff to output buffer; then 

(1 ) Output V„ surpress decoding of the second frame; 

(2) Output B\ allow decoding of the second frame with overwrite of B 1 only as 
memory is freed; 

(3) Output B' again while allowing decoding of the second frame to continue 
overwriting B' as memory is freed; 

(4) Output 6* allow decoding of the third frame with overwrite of B* only as 
memory is freed; 

(5) Output 7* while allowing decoding of the third frame to continue overwriting 
7* as memory is freed; 

(6) Output B 3 , surpress decoding of the fourth frame; 

(7) Output J 3 , allow decoding of the fourth fra me with overwrite of S 3 only as 
memory is freed; ™~~* ™*~~~ 

(8) Output V again while allowing decoding of the fourth frame to continue 
overwriting T 3 as memory is freed; 
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(9) Output V white ailowing decoding of the fifth frame overwriting V as 
memory is freed; 

(10) Output * while aiiowing decoding of the fifth frame to continue overwriting 
. B 4 as memory is freed. 

etc. 

With this output sequence, the use of a buffer memory that is smaiier than the 
3-frame buffer memory of Example 2 is made possible. While the output is not MPEG 
compliant and is of lower quality when used for an entertainment system, the system 
is more economical and produces quality reporduction particularly suitable for many 
industrial applications and low cost consumer systems. 

For the conversions set forth in Examples 3-7 above, similar field substitutions 
of fields can be made to accommodate the use of a 2.53 frame buffer memory. 

As can be seen from Fig. 5. with a 2.53-frame video buffer, the video decoder 56 
must be capable of decoding a slice-row of a frame, particularly macroblock row 30 of 
the first frame, for example, in the time available after the display of row 28 of the top 
field in interval 28. and before the output of row 30 of the top field in interval 30. 
Otherwise, row 30 would have to be skipped. This time can be increased by providing 
additional block rows to the buffer 78. Adding one block row provides a 0.55 frame 
output buffer, or a 2.55 frame video buffer, which lengthens the shortest time interval 
in which the decoder must opera te. Adding three more block rows produces a 0.6 
m ouipui DulTer 0r a *-° Trame ™ eo butter, which further increases the time' 
available for the decoder. Providing the additional time for the decoder ultimately 
makes possible the decoding of the same picture twice. Decoding a frame twice can 
allow the use of a small memory having an output buffer of less than one full frame in 
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size without the need to repface one top or bottom fieid dispiay wfth a disp, ay of the 
other, as in replacing TBT or BTB with TBB or BTT. 

Example 10 

In .he case of 3-2 pulWown co^n. decoding of frames more than once ce„ 
bece.riedou.hce^en^^^^^^ 5uchgs tee ^ 

m>mevideob Uff e,,^e, towi n 9forM P E Gcc^lie n .con»e*gof. fo re Jam p l ea 
motion picture encoded in pro, resslve frames et 24 frames per eeoond to be presented 

bye system having an NTSC display format of 30 frames (60 ne.de) per second, ore 
4:5 frame rete conversion. This example illustrates that, with a video decoder 56 the, 
will decode one frame foster man foe ,/30t„ second display interve!. or a, 1 .5x faster 
foan they are output. This example illusfrates foe, v»here there is time to decode a. 

l.es,eve V ofo*rfran» W ce,nispossfofofo^ea.foe« re ,fta m eofal te ma,ep fc ,u re s 
so that every four frames of a program bitstream in the form on 

T i B',1*B',PB',T'B 4 

can be output as Ave frames in the mannertha, MPEG sped«es. by outputing foe first 
field of the first frame twice as foe first and third fields and foe oufouting „, foe second 
field of foe third frame as foe sixth aod eighth fieids of a ten fieid sequence, as: 

PB'.VB'.TB'.TB'.rB 4 
To output the sequence PB'.TB'rB'. W,rs- in fois case, the dispiay sequence 
control 80 confrols foe decoding and output as follows: sran with decod ing of foe first 
frame end storing V and S' to output buffer, but cause foe DMUX S3 ,o suspend 
ovenvrlting the first frame when it is read from foe video FIFO buffer 67; then: 
(1) Output T, suppress decoding of foe second frame: 
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(2) Output B' ( decode the first frame again with overwrite of B 1 as memory is 
freed, only storing the top field V (because memory is not needed to store 
B 1 again, the decoder can run at 1 .5 block rows per interval, decoding 2/3 
of the frame to be decoded while B' is being displayed); 
5 (3) Output V again while decoding the remainder of V and also decoding 1/3 

of the second frame overwriting V as memory is freed; 

(4) Output B\ decoding the remainder of the second frame, with overwrite of 
B 2 as memory is freed; 

(5) Output 7* while allowing decoding of 2/3 of the third frame, overwriting V 
10 as memory is freed; 

(6) Output B 3 while decoding remainder of third frame and surpressing of 
decoding of the fourth frame; 

(7) Output T 3 , decode the third frame again with overwrite of T as memory is 
freed, only storing the bottom field B 3 decoding 2/3 of the frame to be 

15 decoded while T 9 is being displayed; 

(8) Output B 3 again while allowing decoding of the fourth frame to continue 
overwriting B 3 as memory is freed; 

(9) Output V while allowing decoding of the fifth frame overwriting V as 
memory is freed; 

20 (10) Output B 4 while allowing de coding of the fifth frame to continue overwriting 

B 4 as memory is freed. 

etc. 

A flow chart outlining the operation of the field sequence control logic 80 and the 
picture repeat attribute routine 96 is illustrated in Fig. 6. Upon initiation, the RISC 61 
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sets certain vartab.es based on the configuration of*, host system, inoiudlng .nose 
affecng frame rate conventions, particular*, DRAM memo* size (e. S ., output DUffer 7B 
size = fu.1 frame. 0.53 frame, 0.6 frame, etc.) and the system presented device 
display rate ,30 framesfeeoond for NTSC or 25 frames/second for PAL). When each 
Sequence Header is receded, to. RISC 61 reads from * certain other parameter, 
effecting frame rate oonverston decisions. Including me Progressive Sequence bit end 
the Frame Rate of the encoded original program. The Progressive Sequence bit. found 
in an MPEG-2 sequence extension, spsoilies whether a„ pictures am progressive 
frames that are to be repeated as frames when me pictom header so indicates. The 
original program frame rata may be 25 or 30 frame/saoond standard PAL or NTSC 
sequences or may be 20 or 24 frame/second film sequences mat win require 3:2 pun- 
down conversion. From these vadabtes. a preliminary detem*ation is made in the 
RISC 61 of whether frame rate conversion Is necessao, and. It so, a labia Is contacted 
for use determining which fields am to be repeated or which fields are to be skipped. 

If the dispiay rate of the system dfflem from thai of the sequence of pictums 
being received, the frame rate conversion ratio Is determined and a Counter.Maxhnum 
setting Is made, which eormsponds to toe number of pictums that must be decoded to 
produce a corresponding number of output .elds. For example, to play a 24 frame per 
second pmgressive frame encoded motion picture program on a 30 frame (60 Held) per 
second NTSC interlaced television display, a 4:5 fra me rate conversion ratio is required. 
■ o accompLsn this, live frames (ten felds) must be output for even; tour toll frame 
Pictures received. The Counter.Maximum or Cfr.MAX Is sat at 4. Then.aplc.um 
attribute tabie Is set up which specifies how each picture of a 4 picture recurring 
sequence of pictures is to be displayed. ,„ tola example, toe display of four consecuttve 
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ptetures of the picture sequence may include display of respective top and bottom fields 
of each picture as: TBT, BT, BTB, and TB. So displayed, the top field, bottom field and 
top field again of the first picture are displayed, the top field thereby being displayed 
twice, then the bottom field of the second picture is displayed followed by display of its 
top field, then the bottom field of the third picture is displayed followed by the top field 
of the picture and then the bottom field again, the bottom field thereby being displayed 
twice, and then the fourth picture is displayed top field first then bottom field. This 
produces the desired ten fields of output from the four pictures of the sequence for a 
4:5 frame rate conversion. As a result, by decoding six such 4-picture sequences, 30 
frames of output in one second from 24 frames of original picture input In one second. 

If, based on the variables known at the sequence level, the frame rate of the 
program sequence is the same as that of the system, which is the case where a 25 
frame/second program is being received by a 25 frame/second PAL system and where 
a 30 frame/second program is being received by a 30 frame/second NTSC system, then 
no frame rate conversion is necessary. The Ctr_MAX is set to 0 to designate this 
condition. Where the rates do not so match, the conversion rate ratio is determined as 
follows: 

(1) Where a PAL program (25 frames/sec.) Is being received on an NTSC 
system (30 frames/sec), six frames must be displayed for every five 
received, so the ratio is 5:6 and Ctr_MAX is set at 5. 

(2) Where an NTSC program (30 frames/sec.) is being received on a PAL 
system (25 frames/sec), five frames must be displayed for every six 
received, so the ratio is 6:5. Ctr_MAX is initially set at 6, but one received 
frame is skipped for every five that are'decoded. 
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tS) Where a 24 frame/second program is received (typically a progressive 
frame film program), the program must be converted either to 25 
frames/second for a PAL display or 30 frames/second for an NTSC 
display, respectively calling for 24:25 and 4:5 conversion ratios, typically 
5 using 3:2 pull-down techniques. 

(4) Where a 20 frame/second program Is received (also typically a progressive 
frame film program), the program must be converted either to 25 
frames/second for a PAL display or 30 frames/second for an NTSC 
display, respectively calling for 4:5 and 2:3 conversion ratios, also using 
10 3:2 pull-down techniques. 

When the conversion ratio is determined and the Ctr_MAX variable is set for that 
ratio, a default picture attribute table 98 (Fig. 3) is defined. This table is loaded with 
one picture attribute string for each picture. The string contains bits that tell the field 
display control 82 how to display one or more fields from a given picture. For each of 
15 several possible field display sequences listed in Table 1 below, a hexadecimal code 
is stored in the attribute table. These sequences include those discussed in the 
examples above. The Code in the table represents strings of words of 4-bits each, 
each word representing one field display. The sequence in which the words are used 
for display is from right to left in the table. The four bits of each word include a 
20 rightmost K b n or "B" bit, which is 1 to indicate a bottom (B) field is to be displayed. The 
next bit is the V or "W bit, which is 1 to indicate that, while the field is being read from 
a buffer and displayed, that decoding of another frame is suspended and is to wait The 
third bit is the w d" or "D tt bit, which is 1 when the overwriting of the data in the buffer from 
which the field is being read is disabled so the field data is saved to be displayed again, 
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and is 0 when the data is no .onger needed after being displayed and can be 
overwritten. The 4th bit is the "end' bit, which is only ln the leftmost word of the 
string. 



TABLE I: DEFAULT PICTURE ATTRIBUTE TABLE 



Sequence 


Code 


SdJfl^ edjwb gdjab edwb edwb edwb 


B 


0x000009 


1001 


T 


0x000008 


1000 


BT 


0x000083 


1000 00f1 


TB 


0x000092 


1001 0070 


TT 


0x000082 


1000 0010 


BB 


0x000092 


1001 0011 


BTT 


0x000863 


1000 0110 0071 


BTB 


0x000927 


1001 0110 0071 


TBB 


0x000972 


1001 0111 OOfO 


TBT 


0x000836 


1000 0011 0170 


TBBB 


0x009772 


1001 0111 0111 0070 


TBTB 


0x009276 


1001 0010 0111 0170 


BTTT 


0x008663 


1000 0110 0110 0071 


BTBT 


0x008367 


1000 0011 0110 0171 


TBBBBB 0x977772 


1001 0111 0111 0111 0111 0070 



(bits) 



TBTBTB 0x927676 1001 0010 0111 0110 0111 0170 
BTTTTT 0x866663 1000 0110 0110 0110 0110 0071 
BTBTBT 0x836767 1000 0011 0110 0111 0110 0171 
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ln The d bits in italics are Os in the case of field pictures, since a second field 
decode is needed, so a hexadecimal 2 is subtracted from the Code. The underlined 
italic d bits are Os where a frame picture is to be decoded another time due to memory 
limitations, so a hexadecimal 20 or 2020 is subtracted from the Code. In the table 
above; some of the attribute strings are used for the frame rate conversions described 
above, while others, particularly the longer strings, are used for transitions to and from 
various trick-play modes such as pause and slow forward. 

The picture decode control 81 decodes pictures and keeps track of their 
locations in the buffer memory pending their display. The field display control 82 
displays one field at a time in accordance with four attribute bits and a row map table 
identification it receives from the picture decode control 81. On decoding a picture, the 
decode control 81 constructs and saves an entire one of the attribute strings from 
Table 1 which defines every field output that is to occur from the picture. Whenever a 
field clock signal is received indicating that the next field is to be displayed, the decode 
picture control 81 determines which field is to be displayed and where it is stored in 
memory. It then extracts one four bit word from the attribute string from the picture and 
transmits it along with one row table Identifier 

The value of Ctr_MAX determines the number of consecutive pictures in a series 
that must be decoded and displayed according to a table of attributes to produce the 
desired frame rate conversion. As pictures are received, they are logged in 
presentation order and counted by incrementing a counter from 1 to Ctr_MAX and then 
repeating the cycle as pictures continue to be received and as long as the frame rate 
conversion parameters still apply. Pictures are not necessarily received in their 
presentation orders, so they must be logically rearranged before they are displayed. 
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For example, a typical group of pictures might include pictures intended for dlsp.ay in 

0rd6r 1 12 f0 ' ,0Wln9 *• of ■ Piously decoded reference picture already 
in a buffer 73. The picture types of the twelve pictures of the GOP may be for 
example,., display ordento 12: B, B, I, B, B, P, B, B, P, B, B, P. Reference pictures 
often must be used for copying of data to B-pictures before the reference pictures are 
due for display. As a result, the received order of the pictures may be 3. 1. 2, 6, 4, 5, 
9. 7. 8, 12, 10, 1 1. It is necessary to arrange the picture display attributes in display 
order, not the picture received order, and to correlate the received pictures to the 
corresponding attributes based on theTempommcture^ 
which indicates the relative display order the picture. 

Picture headers and picture extensions contain data that can call for the altering 
of the display attributes originally loaded into the default attribute table 98. Therefore, 
as pictures are received, the attributes are copied from the default attribute table 98 into 
a current attribute table 99 and modified if necessary in the process, if called for by 
Information in the picture data that is received. Fig. 8A illustrates a counter loop by 
which pictures are cycled through the series determined by the Ctr.MAX and attribute 
settings to produce frame rate conversions, if required. When the ioop is first entered, 
the counter is set to 1 and the Current Attribute for the first picture to be received is set 
to a preliminary value indicating Normal display of one field then the other, either TB or 
BT, based on information of the orde r in which pictures have been displayed, or to TB 
. no pr,or order STmown. It at thi s „n ,e a becode^uppre sslfa g ts set, t h ,s rout ine 
waits until the flag is cleared. 

For 6:5 frame rate conversion (from NTSC to PAL), it is only necessary to skip 
one frame and then to display five frames in normal order. Further, all of the standard 
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conversion ratios repeat fields only In the first four frames of the recurring series, 
therefore, when l>4, the normal field display sequence for the picture can be carried 
out. so the normal TB (or BT) attribute is written to the current attribute table 99. 
Otherwise, the default table 98 attribute for the picture is copied to the current attribute 
table 99 for the picture. In any event, the RISC 61 then reads the picture header and 
picture extension, if any. to extract parameter data for use in knowing how to decode 
and where to store the picture, and also how to display the picture. Variables such as 
TFF (top_fieldjirst). P/F (progressivejrame). Frame (frame vs. field DCT) and RFF 
(repeatjirstjield) can. in MPEG-2 pictures, fully determine frame rate conversion from, 
for example 20 or 24 frame/second progressive frame film programs to PAL or NTSC 
display, by. for example, a 3:2 pull-down technique. The picture header data is 
interpreted as set forth in Fig. 6B. Where the resulting display order differs from that 
specified by the default attribute in the default attribute table 99, they supercede the 
default attribute table attribute. In such a case, the default attributes will be modified 
15 when written to the current attribute table 99 for the picture. However, it is possible for 
the system display rate to still be different than that anticipated by the original picture 
parameters, so that even with the program data specifying 3:2 pull-down display order, 
NTSC to PAL or PAL to NTSC conversion must still be made. This is handled by in the 
process of Fig. 6A by adjusting the TFF and RFF variables before calling the routine 
20 of Fig. 6B in the case of 2:3 a nd 24:25 frame rate conversion ratios. 

Fig. 6B illustrates the processing of MPEG-2 picture header and extension data 
that specify field display order, including 3:2 pull-down schemes, contemplated by the 
original program encoder. The determination of field display order based on picture 
header and extension data, as well as sequence header and extension data such as 
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«■ (Passive sequence,, can reauit h ,„ sphy ordere 0 f TB, TBT, BT and BTB 
V*«e the system hee e sma. memory. such as a 0.533 ffame output buffer, B^ctura 
display may also inciude the sequences of TBB end BTT instead of TBT and BTB 
respecdvcy. In ceriain embodiments having the options, aM ^ to decode ^ ^ 
P.cura a second «me where output buffer memory ls ^ ,H an one „ ^ a 
Re.Daooda flag is set end the w-bi, of the appropriate effdbute. Indl ca,ed as -f (en 
underiinedKa,fc 1)ln Tab„ 1. is tetio . 0 , This causes me decode pfchrre conbo, 81 
to signa, the DMUX 53 when the picfore decoded ,o posffton ma read and wri* 
pointer of me video FiFO buffer 67 in me F.FO 83 ofthe DRAM 4 2 to causa decoding 
of seme ffame picbae me ne« „me a pfcfore ls to be decoded . ^ ^ ^ 
emibutes for a picture have been so determined, may as soon as buffer memo* for 
sfortng the decoded picture becomes eveiiebte, me plcfom le decoded and stored in a 
buffer 73 or 78. 

In Table 1. attributes are provided for ma repetition of fields in various ways for 
use in certain frtck piay modes, such as for transitioning Mo and out o, a pause 
condMon or Into and out of a slow forward condition. 

Those skilled in the art wil. appreciate that the applications of the present 
invention herein are varied, and ma. me Invent is described in preferred 
embodiments. Accordingly, additions and modificalions can ba made without departing 
from me principles of me invenbon. What is claimed is: 
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1. A method of decoding and drying a sequence o, digite, video p.curea 
comprising the steps of: 

-ponding ,o interna ,n an input video biteuean, ,o determine feme rate 
conversion needed to display pictures of the sequence; 

determining the amoun. buffer memory avaliabia for use in dispiaylng the 
sequence; and 

in response ,o me determined tame rate conversion and the detemtined 
ava„ab,e memory, defining dirW, fteid display sequences «o accommodate different 
memory availability determinations and feme rate conversions. 



2- The method of claim 1 wherein: 

the fiatd display sequence denning step Includes radispraylng , he nret ^ ot 
selected femes again after displey of ^ ^ ^ ^ ^ ^ ^ 

memory is one u feme in see and radispiayfc* me second field of selected femes 
aga n afterd l sp,ayof tt ,esecondfe k , W nerea»amoun,o,buffarmemory,s,essu,an 



one full frame in size. 



3. The method of claim 1 further comprising the steps of: 

establishing a recurring sequence count based on the determined frame rate 
conversion and decoding recurring sequences of n octures to produce recurring series 
of decoded fieid pictures, displaying selected field pictures of the series more than once 
to implement the determined frame rate conversion. 



method of claim 1 further comprising the step of: 
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conditioning the decoding step on the availability of buffer memory on a block 
row basis. 



5. A digital video decoder programmed to perform the method of claim 1 . 
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6. A method of decoding an MPEG video signal comprising the steps of: 
providing display logic configured to output a next field of digital video from 

designated locations in a buffer memory; 

decoding logic picture data from a coded sequence of pictures and storing fields 
of digital video of the decoded picture to available locations in the buffer memory; 

converting the frame rate of decoded pictures for display by determining 
(a) whether 3:2 pull-down is indicated, (b) whether NTSC/PAL conversion is indicated 
and (c) whether the buffer memory is sufficiently large to store one full frame of 
decoded output, and 

displaying the frame pictures as a sequence of fields at a frame rate converted 
in accordance with the determinations. 

7. The method of claim 6 wherein: 

the determination of whether 3:2 pull-down is indicated is responsive to 
information accompanying the picture data where the data is of a DVD program. 

8. The method of claim 6 wherein: 

the determination of whether 3:2 pull-down is indicated is responsive to a 
comparison of the format of the input with the configuration of the display. 
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9. The method of claim 6 further comprising the steps of: 

associating field sequence display attribute variables with each of the decoded 
• picture fields to coordinate the freeing of buffer memory as fields are displayed for use 
storage upon decoding of a subsequent frame picture and the timing of the decoder to 
accommodate field repetition. 



>. A digital video decoder programmed to perform the method of claim 6. 
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11. An MPEG digital video presentation apparatus comprising: 

a display logic section responsive to a "next field" command to output a field of 
digital video from designated locations in a buffer memory; 

a frame decoding section operative to decode picture data in response to a "next 
Picture" command and to store fields of digital video of the decoded picture data into 
available locations in the buffer memory; 

a frame rate determination module logically connected between the display logic 
section and the frame decoding section determining frame rate conversion 
requirements and producing field display sequence management to implement the 
determined frame rate conversion requirements. 
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